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AMENDMENTS TO THE DRAWINGS 

Please add the attached one (1) new sheet of drawings containing new Fig. 2. 



Attachment: One (1) new sheet of drawings 
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REMARKS 



The present response is intended to be fully responsive to all points of 
objection and/or rejection raised by the Examiner and is believed to place the 
application in condition for allowance. Applicants assert that the present invention is 
new, non-obvious and useful Prompt consideration and allowance of the claims is 
respectfully requested. 

Status of Claims 

Claims 1-33 are pending in the application, with claims 13-20, 22, 23 and 33 
having been withdrawn from consideration. Claims 1-12, 2 land 24-32 have been 
rejected. 



The Drawings were objected to under 37 CFR 1.83(a) for not showing every 
feature of the invention specified in the claims. Specifically, the Examiner states that 
the drawings do not show the step of "determining the membership". Corrected 
drawing sheets in compliance with 37 CFR 1.121(d) are required in the replay to the 
office action to avoid abandonment of this application. 

In response, Applicants herewith submit a new drawing (Figure 2) on a new 
sheet. The entire drawing sheet containing the new drawing Figure 2 is enclosed for 
review by the Examiner.. This drawing .generally repeats in the form of a flow 
diagram the method steps recited in claim 1 and does not add matter. 



In the final Office Action, the Examiner rejected claims 1-4, 6-8, 12, 24-27, 
and 29-32 under 35 U.S.C. § 103(a), as being unpatentable over Castango et al. (IEEE 
Vol. 8, No. 5, Sep. 1998, 562-571), Park et al. (U.S. Patent No. 6,535,632) and 
Bierling et al. (U.S. Patent No. 4,771,331). 



Drawings Rejections 



CLAIM REJECTIONS 



35 U.S.C § 103 Rejections 
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The Examiner also rejected claim 5 under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Castangno, Park, and Bierling as applied to claim 
1 discussed above, and further in view of Aggarwal et al. (U.S. Patent No. 6,728,706). 

The Examiner also rejected claims 9-11 under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Castagno, Park, and Bierling as applied to claim 1 
discussed above, and further in view of Price et al. (U.S. Patent No. 5,606,164). 

The Examiner also rejected claim 28 under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Castagno, Park, and Bierling as applied to claims 
1, 25, 26 and 27 discussed above, and further in view of Penn (U.S. Patent No. 
5,848,198). 

Applicants traverse these rejections in view of the remarks that follow. 

The present invention provides a method for segmenting an image which is 
formed of pixels. At each pixel, there are feature values which include pixel values 
(such as R, G and B values) and motion vector values (such as the horizontal and 
vertical components of a motion vector expressed in pixels per frame period). The 
data is represented in a segmentation vector space which is the product of the vector 
space of those feature values and the vector space of pixel addresses. Segments are 
then represented as locations in this segmentation vector space. The membership of a 
segment for each pixel is then determined by the distance in segmentation vector 
space from the data point representing the pixel to the location of that segment. 

It may be helpful here to take a numeric example using the example value 
quoted above and employing the methodology described in the present patent 
application. Thus, if: 

H is the horizontal spatial coordinate of the pixel 
V is the vertical spatial coordinate of the pixel 
R is the red component of the pixel 
G is the green component of the pixel 
B is the blue component of the pixel 

D H is the horizontal component of the motion (or displacement) vector 
D v is the vertical component of the motion (or displacement) vector, 

then the location of a segment in the segmentation vector space will have coordinates 

in each of these dimensions and may take the value: 
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Hh> I^Vs |*r, |*g> Hb, I^dh, Hot- 

In an initialization step, these coordinates defining the location of the segment 
will be pre-set In subsequent iterations, the location of the segment will typically be 
calculated as the centroid in segmentation vector space of all pixels belonging to that 
segment. 

The membership of a segment for each pixel is then determined by a distance 
in segmentation vector space from the data point representing the pixel to the location 
of a segment, and an example of such a distance might be: 

(H-M 2 + (V- fiv) 2 

+ 0.5(R - Hr) 2 +1.5(G - \x G ? + 0.3(B - jx B ) 2 
+ 10(Dh~(Xdh) 2 + 10(D v -^dv) 2 

The scaling factors used in this example are taken from the formula at page 6, 
line 5 of the present application as filed (paragraph [0032] of the application as 
published). The smaller this distance, the "closer" is a pixel to the centroid of a 
segment and the more likely it is to be regarded as a member of that segment. 

Looking at this formula it will be recognized that a pixel will tend to be 
included in the membership of a segment if: 

? It is spatially close to the segment, that is to say its H and V values are 

close to the corresponding value of the segment. 
? Its pixels value are close to those of the segment, that is to say its. R, G 

and B values are close to the corresponding values of the segment. 
? It has the same motion vector as the segment, that is to say its D H and D v 

values are close to the corresponding values of the segment. 

This is reasonably intuitive. If two pixels in an image are spatially close, if they have 
similar color values, and if they are moving in the same direction, then they are likely 
to be representing the same object and should indeed form part of the same segment. 

The Examiner has concluded that the features of the invention as claimed and 
as so far described are obvious based upon a combination of Castagno and Park. 
However, the feature of the claimed invention not yet described and the feature of the 
invention which the Examiner does not find in the combination of Castagno and Park 
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is "aaid distance in segmentation vector space including a displaced frame difference 
calculated by applying a motion vector from the segment to the pixel". 

Thus, in accordance with the invention as claimed, there would be added to 
the distance measure of the example quoted above, a further value which is the 
displaced frame difference or DFD. An example of a distance measure may then be: 

(H-ji H ) 2 + (V^ v ) 2 

+ 0.5(R - fi R ) 2 +1.5(G - \i G ) 2 + 0.3(B - n B ) 2 
+ 10(D h ^dh) 2 + 10(Dv- |*dv) 2 
+ 2DFD 2 

At first glance this seems a very strange step to take. It has been noted above 
that pixels which are spatially close, which have the same or similar RGB values and 
which have the same or similar motion might intuitively be regarded as likely to form 
part of the same segment. It can therefore be understood why those values appear in 
the distance measure. On the other hand, a DFD value is by-product of the technique 
employed for calculating the displacement vector. The DFD value will be zero or 
small if the displacement vector is accurate and will be larger if the displacement 
vector is less accurate. There is no intuitive reason why any two pixels which have the 
same DFD value should be likely to belong to the same segment. It is, on this basis, a 
very odd thing indeed to include a DFD value in the distance measurement. 

The present inventors have recognized, however, that the DFD value is a 
measure of the fidelity of the displacement vector measurement (see page 10, line 5 of 
the present application as filed (paragraph [0050] of the application as published) and 
that by including the DFD value in the distance measure, the segmentation process 
can be made to take proper account of the fidelity of the motion vector information. 
Note that, as described in the above example, a pixel which is close to a segment in 
the chosen dimensions will be likely to be regarded as a member of that segment. 
There may be two pixels which are candidates for membership of a particular segment 
and which have very similar values of these coordinates. In one case, however, the 
fidelity of the displacement vector calculation is high (and the DFD value is low) and 
in the other case the fidelity of the displacement vector derivation is low (and the 
corresponding DFD value is high). 
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The present invention, by including the DFD value in the distance 
measurement itself now achieves the desired goal that the pixel value with high 
fidelity in the displacement vector is more likely to be included in the segment than is 
the pixel value with low fidelity. The distance measurement for the pixel with low 
fidelity in the displacement vector will be greater than the distance measurement of 
the pixel within high fidelity. 

As will be described in more detail below, this ingenious result cannot be 
derived from any obvious combination of Castagno and Park and Bierling. 

The Examiner's 103(a) rejection of claim 1 relies on the Examiner's finding 
that it would have been obvious at the time the invention was made to one of ordinary 
skill in the art to be motivated to include the segmentation system of the Castagno and 
Park combination with displaced frame difference (DFD) as one of the feature values 
as taught by Bierling in order to "further reduces the jerkiness in the motion 
compensating interpolated sequence" (the Examiner referred to column 3, line 28 of 
Bierling). 

The reasons why Applicant submits that that this finding by the Examiner is 
incorrect are as follows: 

1. The motivation cited by the Examiner lacks technical foundation. 

The Examiner explains that the motivation to include the DFD in the 
segmentation of Castagno and Park is to "further reduce the jerkiness and the motion 
compensating interpolated sequence". The Examiner quotes this language from 
column 3, lines 28 of Bierling, where it is stated that "this further reduces the 
jerkiness in the motion compensating interpolated sequence". What is referred to by 
the term "this" is set forth in the remainder of the paragraph at column 3, lines 16-36. 
Thus, what "further reduces the jerkiness" is "the hierarchically structured 
displacement estimation technique". Thus, if one of ordinary skill in the art reading 
Bierling was minded to reduce the jerkiness in a motion compensating interpolated 
sequence, he is instructed to use displacement vectors which are generated by the 
displacement estimation technique disclosed in Bierling. The reduction of jerkiness 
arises from the displacement estimation technique and specifically by the feature of 
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the technique that it provides uniquely defined displacement vector fields (column 3, 
lines 25 to 26). 

2. A combination of Bierling with Castagno and Park would be outside the claim 
scope. 

If one of ordinary skill in the art reading (despite any teaching or motivation to 
do so) were instructed to combine Bierling with Castagno and Park, he would use the 
DFD of Bierling in an improved algorithm for calculating the motion or displacement 
vectors in Castagno. Such a combination would not result in a distance in 
segmentation vector space including DFD. 

3. The use of DFD in the claimed invention is not analogous with the use of DFD in 
Bierling. 

As has been noted above, Bierling uses DFD values in the derivation of 
displacement vectors. The accuracy of the displacement vectors generated by Bierling 
is improved by minimizing the DFD value. In Bierling, once the hierarchically 
structured displacement estimation technique has produced displacement vectors, the 
DFD values serve no further purpose. 

In the present invention, the DFD's are used alongside the displacement vector 
values. In the present invention, the DFD's are not used as a parameter which is 
minimized in improving the accuracy of the displacement vector but as part of a 
distance measurement used in defining segment measurement for the pixel. 

For the reasons set forth above, the Examiner is respectfully requested to 
reconsider and withdraw the rejection of independent claim 1. Claims 2-12 and 24-32 
are dependent upon claim 1 and include all the limitations thereof, such that these 
claims are patentable at least by virtue of their dependence from claim L 

The Examiner also rejected claim 21 under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Castango, Park, and Bierling as applied to claim 
1 discussed above, and further in view of Globus et al. (U.S. Patent No. 4,078,860). 
The Examiner has found that it would have been obvious to one of ordinary skill in 
the art to include in the segmentation system of the Castagno, Park and Bierling 
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combination a toroidal canvas as taught by Globus, such that (referring to column 1, 
line 41 of Globus) "a non-distorted and reliable image to be produced in a completely 
circular mode". 

According to independent claim 21, the segmentation vector space in which 
the image data is represented as a toroidal canvas. This solves the problem of the 
disappearance and re-appearance of objects because of global motion in the scene. 

Applicants first argue that Globus does not disclose a toroidal canvas. Globus 
discloses a toroidal convex lens and a circular screen. There is nothing in the teaching 
of Globus which is remotely analogous to the claim integer of representing the image 
data as points in a segmentation vector space in a toroidal canvas. 

Applicants also argue that Globus is concerned with image projection. If one 
of ordinary skill in the art required that "a non-distorted and reliable image to be 
produced in a completely circular mode" he would employ the projection system of 
Globus in the display of a video signal. 

Even if that video signal had previously undergone processing in accordance 
with the teaching of the Castagno, Park and Bierling combination, there would be no 
method falling within the scope of independent claim 21. Accordingly, Applicants 
contend that independent claim 21 is not obvious over the combination of Castango, 
Park, and Bierling in view of Globus et al. 

The Examiner has helpfully set out in some detail his responses to various 
remarks of Applicants in the previous reply. Applicants hereinbelow provide their 
comments, conveniently referring to the Examiner's responses, retaining the 
alphabetic headings employed by the Examiner. 

(a) The withdrawal of these previous objections is noted with appreciation. 

(b) The Examiner has asked for further clarification of the meaning of the terms: 

(a) motion vector; 

(b) displacement vector, and 
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(c) displaced frame difference. 



In this context, the terms motion vector and displacement vector may be 
regarded as synonymous. The Examiner has correctly identified the vector "D" in 
Figure 2 of Bierling as an example of a motion or displacement vector. 

A displaced frame difference, as noted by Bierling at column 5, line 64, is the 
frame difference which remains after displacement by the displacement vector. The 
displaced frame difference (DFD) approaches zero as the estimated displacement 
vector approaches a true displacement vector. The accuracy of the displacement 
vector is improved in a process which minimizes the displaced frame difference. 
Thus, the DFD can be seen to be a parameter which is used in generation of the 
motion or displacement vector and which will be zero if that motion or displacement 
vector is precise. 

(c) The Examiner's response to Applicants' remark is to refer to passages in 
Castagno which mention spatio-temporal segmentation using motion information and 
to refer to a passage in Bierling which states that the displacement vector technique of 
Bierling provides vector fields which are valid for the temporal positions of the field 
to be interpolated. Applicants respectfully submit that these passages are entirely 
consistent with the explanations given previously in this response of the teaching of 
Castagno and of Bierling and do not suggest any combination of the references which 
would fall within the claim scope. 

Castagno does indeed use motion information in its segmentation. That motion 
information takes the form of displacement vectors. Bierling teaches a technique for 
the measurement of displacement vectors, that technique including the use of 
displaced frame differences. If the technique of Bierling were to be used in 
combination with the teaching of Castagno, the effect would be to use the improved 
displacement vectors of Bierling in place of the displacement vectors mentioned by 
Castagno. Applicants repeat that there is nothing in Bierling, in Castagno or in Park 
that suggests that the DFD value mentioned in Bierling should be included in a 
distance in segmentation vector space in an image segmentation process. 
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(d) The Examiner observes that a zero DGD is the expectation for unchanged 
picture areas. This is, with respect to the Examiner, a misreading of Bierling. The 
passage at column 2 of Bierling which includes line 27 categorizes the method as of 
the type wherein a displacement vector is generated by an iteration process for each 
picture element of the field to be interpolated and zero displacement vectors are aside 
the picture elements in unchanged picture areas by means of a change detector. The 
DFD is used in the first step of this process in the generation of a displacement vector 
by an interaction process for each picture element of the field to be interpolated. In 
this process, it is correct to say that Bierling is always striving to reduce the DFD to 
zero. It is further true to say that a non-zero DFD is in the context of Bierling merely 
an indication that the correct displacement vector has not yet been achieved. 

In the second stage of the process, Bierling refers to zero displacement vectors 
being assigned to picture elements in unchanged picture areas by means of a change 
detector. It is important to note here that Bierling is speaking of zero displacement 
vectors and not zero DFDs . 

(e) The Examiner has stated that the correct test is what the combined teachings 
of the references would have suggested to those of ordinary skill in the art. Applicants 
respectfully remark that the teaching of Globus is the use of a toroidal lens to produce 
a circular image . It is fanciful to suggest that the mention of toroidal lens in Globus 
would be taken by the skilled reader as a suggestion that the mathematical 
segmentation vector space in Castagno and Park should be given a toroidal canvas. 

(f) The meaning in claim 21 of the term "toroidal canvas" would be clear to one 
of ordinary skill in the art without the need to read limitations from the specification. 
The claim limitation of representing the image data of points in a segmentation vector 
space in a toroidal canvas is clear and it is clear that the claim limitation is not met by 
displaying a visual image through a toroidal lens. 

In view of the foregoing amendments and remarks, the pending claims are 
deemed to be allowable. Their favorable reconsideration and allowance is respectfully 
requested. 
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Should the Examiner have any question or comment as to the form, content or 
entry of this Amendment, the Examiner is requested to contact the undersigned at the 
telephone number below. Similarly, if there are any further issues yet to be resolved 
to advance the prosecution of this application to issue, the Examiner is requested to 
telephone the undersigned counsel. 

Please charge any fees associated with this paper to deposit account No. 50- 



Dated: January 30, 2009 

Pearl Cohen Zedek Latzer, LLP 

1500 Broadway, 12th Floor 
New York, New York, 10036 
Tel: (646) 878-0800 
Fax: (646) 878-0801 



3355. 



Respectfully submitted, 




Morey B. Wildes 
Attorney/Agent for Applicant(s) 
Registration No. 36,968 
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